Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation
نویسندگان
چکیده
منابع مشابه
Adaptive Importance Sampling Technique for Markov Chains Using Stochastic Approximation
For a discrete-time finite-state Markov chain, we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states. This scheme updates the change of measure at every transition using constant or decreasing step-size stochastic approximation. The updates are shown to concentrate asymptotically in a neighborhood of the desired zero varian...
متن کاملAdaptive Importance Sampling on Discrete Markov Chains
In modeling particle transport through a medium, the path of a particle behaves as a transient Markov Chain. We are interested in characteristics of the particle's movement conditional on its starting state which take the form of a \score" accumulated with each transition. Importance sampling is an essential variance reduction technique in this setting, and we provide an adaptive (iteratively u...
متن کاملAdaptive Importance Sampling for Uniformly Recurrent Markov Chains
Importance sampling is a variance reduction technique for efficient estimation of rare-event probabilities by Monte Carlo. In standard importance sampling schemes, the system is simulated using an a priori fixed change of measure suggested by a large deviation lower bound analysis. Recent work, however, has suggested that such schemes do not work well in many situations. In this paper, we consi...
متن کاملHonest Importance Sampling with Multiple Markov Chains.
Importance sampling is a classical Monte Carlo technique in which a random sample from one probability density, π1, is used to estimate an expectation with respect to another, π. The importance sampling estimator is strongly consistent and, as long as two simple moment conditions are satisfied, it obeys a central limit theorem (CLT). Moreover, there is a simple consistent estimator for the asym...
متن کاملAn Adaptive Importance Sampling Technique
This paper proposes a new adaptive importance sampling (AIS) technique for approximate evaluation of multidimensional integrals. Whereas known AIS algorithms try to find a sampling density that is approximately proportional to the integrand, our algorithm aims directly at the minimization of the variance of the sample average estimate. Our algorithm uses piecewise constant sampling densities, w...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Operations Research
سال: 2006
ISSN: 0030-364X,1526-5463
DOI: 10.1287/opre.1060.0291